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ON  TBE  NONUNEAH  RENEWAL  TBEOBEM^ 


By  Michael  WooDBOOFE 

ITie  Unioenity  ofMirJtigan 

Let  Zi,Zs,...  be  jointtjr  dietribiitad  midoai  miaUM  for  which  ntpt 
f.  Z/k  -  •  w.p.1  and  kt  I  ~  *  inf(n  2  1;  Z„  >  a)  and  <■  Z^  o  for 

e  0.  Coikhtiau  under  ^^liA  A,  haa  a  limitinc  diabSmtion  as  a -*  •  an 
devaiopsd.  Tbaae  raquiia  that  the  finite  dfananaional,  conditional  distribu* 
thms  of  tha  ineronMnte  Zf^.*  -2y,  A  2  1,  conwria  to  tba  finite  dimen* 
temal  diatributiona  of  a  proeaaa  fiar  edddi  tba  laauit  is  known,  thus 
wsokaninc  tha  alow  dianga  condition  in  aariiar  work.  Hm  main  result  is 
^pBad  to  aoma  aaquanoes  for  wfaidi  tha  linitinf  distributions  are  those  of 
the  partial  sums  of  an  aidiangaahle  proeaaa.  Ihasa  induds  the  Eudidean 
noma  of  a  driftlaaa  random  walk  in  aeaeral  (iimenaiona  and  aaquancea  for 
iriikfa  tha  conditional  distribution  of  Z„^i  -  given  tha  past  hu  a  limit 
w.p.l  as  n  -*  «. 


L  Liirodiiotion.  Let  (O,  J3^,  P)  denote  a  probability  space,  let  s 
Jir,  S  ...  denote  subsigma-algebras  of  Jif  and  let  Z^,  Z2, . . .  be  random  vari¬ 
ables,  defined  on  (n,,!/,P),  for  which  Zj^  is  Jifji  measurable  for  all  k  «  1,2,... 
and 


(1)  SUpZjt  «  CO  w.p.l. 

Such  a  sequence  miQr  be  called  an  infinite  supremum  process.  For  any  such 
process,  the  first  passage  times  and  excesses 


(2)  f.  -  inffA  ^  1:  Zj  >  0} 
and 

(3) 

may  be  defined  for  all  a  ^  0  w.p.l.  Let  denote  the  distribution  function  of 
R^,  that  is. 


(4)  H.(r) -P{fa  <oo,i?„  <;r),  Va,r^0. 

The  problem  considered  is  to  find  conditions  imder  which  R^  has  a  limiting 
ihitribution  H  sm  a -*(*>•,  that  is,  H  as  a  -» co,  where  ^  denotes  weak 


lUottvtd  hlay  1968;  rtviMd  Junt  1989. 

^RitMTch  iupport«d  the  NtiioRtl  Sdenoe  Foundation  under  Grant  DMS*84-13452  and  the 
U.S.  Army  Reeearch  Office  under  Grant  DAAL03-88-0122. 
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ONE-ABMED  BANDIT  PROBLEMS  WITH  COVARIATES* 

By  JYOTIBliOY  Sabkab 
Univenity  of  Michigcm 

Am  doM  Woodroofe,  we  eoneider  a  Bagpeeiaii  eequential  allocation  be* 
tween  two  treatmenta  that  inootporatae  a  covariate.  The  goal  ia  to  maxi* 
mize  the  total  diaoounted  eaqtected  reward  fiton  an  infinite  population  of 
patienta.  Althou^  our  modd  ia  more  general  than  Woodroofe’a,  we  are 
able  to  duplicate  hia  main  reault:  The  myopie  rule  ia  aaymptoticalljr  opti* 
maL 


1.  Introdnction. 

1.1.  Statement  of  the  problem.  Consider  a  population  of  patients  who 
arrive  seqpientially  for  treatment  of  a  disease.  Suppose  each  pi^ent  may  be 
treated  with  either  a  standard  treatment  vdiose  statistical  characteristics  are 
known,  or  a  new  treatment  udiose  characteristics  are  unknown.  Also  suppose 
that  before  deciding  to  assign  a  given  patient  to  a  treatment,  we  obswve  a 
covariate  X,  such  as  age,  severity  of  disease  or  general  physical  status,  which 
is  specific  to  the  patient. 

1^  y**  and  y*  denote  the  potential  rewards  firom  the  standard  and  the  new 
treatment,  respectively.  Let  5  >-  0  and  5  <■  1  denote  the  choice  of  standard  or 
new  treatment  respective.  We  would  like  to  assign  patients  to  treatments  in 
such  a  manner  that  the  total  discounted  expected  reward  over  the  vdiole 
population  of  patients  is  maximized.  The  diacoimt  sequence  is  geometric  with 
di^unt  factor  a  €  (0, 1). 

1.2.  Rationale  for  covariate  model.  In  clinical  trials,  the  goals  an  experi¬ 
menter  would  like  to  attain  are  diverse  and  often  conflicting.  Ethical  consider¬ 
ations  are  prominent  in  all  experimentation  involving  human  subjects. 
Conflicts  are  invariably  generated  by  the  obligation  of  a  researcher  to  balance 
the  well-being  of  the  current  patient  (individualistic  view)  with  that  of  the 
future  patients  (utilitarian  view)  who  stand  to  benefit  from  new  advances  in 
medical  treatment.  This  long-standing  dilemma  has  received  considerable 
attention  in  both  statistical  and  medical  literature  such  as  Anscombe  (1963), 
Weinstein  (1974),  Byar,  Simon,  Friedevmld,  DeMets,  Ellenberg,  Gail  and  Ware 
(1976),  Bartlett,  Roloif,  Cornell,  Andrews,  Dillon  and  Zwischenberger  (1985) 
and  Woodroofe  and  Hardwick  (1990). 


Raceived  November  1989;  revised  January  1991. 

^Research  supported  by  IJ.S.  Army  Reaemh  Office  under  DAAL  03-88-0122  and  NSF  under 
DMS  89*02188. 

AMS  1980  subject  classification.  62L10. 

Key  wo^  and  phrases.  Sequential  allocation,  one*armed  bandit  problem,  Bayesian  analysis* 
regret,  policy,  aiymptotically  optimal. 
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SEQUENTIAL  ANALYSIS,  9(4),  335*342  (1990) 


ON  STOPPING  TINES  AND  STOCBASHC  NONOTQNICZTY 
By 

Michael  Woodroofe 
The  University  of  Michigan 

Key  words  and  phrases :  est  imtion  after  stquentiMl  test  ing: 
exponentlel  fajsiiies;  the  fundeaetitel  identity  of  sequentiel 
enelysis* 


ABSTRACT 

If  t  is  any  stopping  tine,  then  the  distributions  of  are 
stochastically  iM>notone  in  the  paraaeter,  when  saapling  from  a  one 
parameter  exponential  faaily.  The  proof  is  a  staple  exercise  in 
differentiating  the  fundamental  Identity  of  sequential  analysis. 
Some  other  applications  of  the  technique  are  included. 


1.  INTRODDCnOH> . 

Siegmund  (1978)  has  suggested  an  ingenious  method  for  setting 
confidence  intervals  after  sequential  testing,  which  involves 
ordering  boundai7  points  in  a  counter  cloclcwlse  direction  and  then 
inverting  a  collection  of  tests.  See  also  Siegmund  (1985, 
Sections  3.4  and  4.5).  Recently,  Bather  (1988)  has  oerived  a 
stochastic  aonotonicity  property  for  the  counter  clockwise 
ordering  for  generalized  sequential  probability  ratio  tests  and 
exponential  families. 

An  al ternat ive  approach  to  est imat ion  after  sequent ial 
testing  Is  to  estimate  the  parameter  by  Its  maximum  likelihood 
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INTEGRABLE  EXPANSIONS  FOR  POSTERIOR 
DISTRIBUTIONS  FOR  ONE-PARAMETER 
EXPONENTIAJ  FAMILIES 


Michael  Woodroofe 
University  of  Michigan 


Ahstmet:  Tht  m>iB  laialta  ptovidt  asymptotic  expansions  fox  posterior  distributions 
which  may  be  integrated  termwise  with  respect  to  the  marginal  distribution  of  the 
data.  The  proof  uses  a  data  dependent  transformation  which  converts  the  likelihood 
function  to  exact  normality  and  then  applies  a  version  of  Stein’s  Identity  to  the 
posterior  distributions.  Applications  to  sequential  confidence  intervals  are  described 
briefly. 

Key  words  and  phrases;  Posterior  distributions,  parameter  transformations,  Stein’s 
Identity,  martingale  convergence  theorem,  stopping  times,  sequential  confidence  in¬ 
tervals. 


1.  Introduction 

Asymptotic  expansions  for  posterior  distributions  may  be  traced  to  the  time 
of  Laplace,  but  only  recently  have  researchers  investigated  conditions  under  which 
thes'i  expansions  may  be  integrated  with  respect  to  the  marginal  distribution  of 
the  data.  See  Johnson  (1970)  for  a  rigorous  account  of  the  pointwise  expansions 
and  Bickel,  Goetze  and  Van  Zwet  (1985)  and  Ghosh,  Sinha  and  Joshi  (1983)  for 
recent  work  on  integrating  them. 

In  this  article,  an  alternative  approach  to  the  expansions  is  presented  which 
makes  the  question  of  integrability  more  transparent.  First,  instead  of  renormal¬ 
ized  estimation  error,  a  data  dependent  transformation  (S'  of  the  parameter  is 
considered,  which  converts  the  likelihood  to  exact  normality.  Then  a  version  of 
Stein’s  Identity  is  applied  to  the  posterior  distributions  to  isolate  the  remainder 
terms.  The  alternative  approach  avoids  vhe  use  of  messy  Taylor  sent  expansions 
and  leaves  the  renormalized  remainder  terms  in  the  form  of  conditiona.  expecta¬ 
tions,  so  that  the  martingale  theory  may  be  brought  to  bear  on  the  integrability 
question. 

Integrable  expansions  for  posterior  distributions  are  needed  in  design  prob¬ 
lems  where  the  overall  Bayes  risk  must  be  computed  in  order  to  see  the  oTect 


AN  ADAPTIVE  BIASED  COIN  DESIGN 
FOR  THE 

BEHRENS-FISHER  PROBLEM 


Jeffrey  R.  Eiseie 

Department  of  Statistics 
The  University  of  Michigan 
Ann  Arbor,  MI  48109 

Key  words  and  Phrases:  asymptotic  optimality;  experimenter  bias;  sampling 
plan;  sequential  design;  stopping  rule. 


ABSTRACT 

W&i(1978)  introduced  the  adaptive  biased  coin  design  to  reduce  experi¬ 
menter  bias  and  offer  a  compromise  between  peifect  balance  and  complete 
randomization.  In  situations  such  as  the  Behrens-Fisher  problem,  balance  is 
not  necessarily  desired  and  the  optimal  ratio  of  sample  sizes  is  unknown.  To 
reduce  experimenter  bias,  by  introducing  randomization,  an  adaptive  biased 
coin  design  is  superimposed  on  Robbins,  Simons,  and  Starr *s(  1967)  sequen¬ 
tial  ajialogue  of  the  Behrens-Fisher  problem.  The  design  has  asymptotic 
properties  similar  to  Robbins,  Simons,  and  Starr’s  sequential  procedure. 


1.  INTRODUCTION 

Suppose  patients  arrive  sequentially  at  an  experimental  site  and  are  assigned 
immediately  to  one  of  two  treatment  groups  A  or  B.  It  is  desired  to  de¬ 
sign  a  sequential  procedure,  with  a  randomized  allocation  scheme,  for  the 
fixed  width  interval  estimation  of  the  difference  of  the  means  of  these  two 
populations. 
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NONLINEAR  MARKOV  RENEWAL  THEORY 
WITH  STATISTICAL  APPLICATIONS 
By 

Vincent  F.  Meifi 
The  University  of  Michigan 


ABSTPJ^CT 

An  analogue  of  the  Lai-Siegmund  nonlinear  renewal  theorem  is 
proved  for  processes  of  the  form  5r  +  where  {5n}  is  a  Markov 
random  walk.  Specifically,  Vo,  Yi, ...  is  a  Markov  chain  with  complete 
separable  metric  state  space;  Xi ,  X2, . . .  is  a  sequence  of  random  vari¬ 
ables  such  that  the  distribution  of  Xi  given  j  >0  and  Xj^  j  ^  i 
depends  only  on  Y-i  and  VJ;  Sr  =  -Yi  — 1-.Yr;  and  {^r  }  is  slowly 
changing,  in  a  sense  to  be  made  precise  below.  Applications  to  se¬ 
quential  analysis  are  given  with  both  countable  and  uncountable  state 
space. 


Key  toords  and  phrases.  Markov  chain,  Markov  random  walk,  nonlinear  renewal 
theorem,  excess  over  the  boundary,  repeated  significance  test. 

AMS(1980)  Classifications.  Primary  60K05,  60K15;  secondary  62L10,  60J10. 
Research  supported  by  the  U.S.  army  under  DAAL  03-88-0122 
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Estimation  after  sequential  testing:  a  simple  approach  for  a 
truncated  sequential  probability  ratio  test 

By  MICHAEL  WOODROOFE 

Statistics  Department,  The  University  of  Michigan,  Ann  Arbor,  Michigan  48109,  U.S.A. 

Summary 

An  approximate  pivot  is  constructed  for  the  problem  of  estimating  a  normal  mean  6 
following  a  truncated  sequential  probability  ratio  test  and  shown  to  provide  a  useful 
method  for  constructing  confidence  bounds  and  intervals.  Letting  t  denote  the  sample 
size  and  S,  the  sum  of  observations,  the  approximate  pivot  is  constructed  by  standardizing 
Sf-t~^(S,-t0)  the  mean  and  variance  of  which  are  no  longer  0  and  1,  due  to  the 
optional  stopping.  The  truncation  of  the  sequential  probability  ratio  test  is  done  in  a 
nonstandard  way  in  order  to  smooth  the  boundary. 

Some  key  words:  Asymptotic  normality;  Confidence  levels;  Error  probabilities;  Expe^ed  sample  size;  Means 
and  variances;  Simulation. 


1.  Introduction 

Sequential  tests  have  long  been  advocated  as  a  means  of  reducing  the  ethical  problems 
inherent  in  randomized  clinical  trials  on  human  patients.  There  are  now  several  good 
sequential  tests  which  are  finding  increasingly  widespread  applications.  See,  for  example, 
the  books  by  Armitage  ( 197S),  Siegmund  (1985)  and  Whitehead  ( 1983)  for  complementary 
accounts  of  these  developments. 

Of  course,  the  use  of  a  sequential  test  complicates  the  problem  of  estimating  parameters, 
after  the  test  has  been  concluded.  In  response  to  this  problem,  Armitage  (1957)  proposed 
a  confidence  procedure  which  explicitly  incorporates  the  stopping  rule  into  the  calculation 
of  coverage  probabilities  for  binary  data;  and  the  proposal  was  developed  in  some  detail 
by  Siegmund  (1978)  for  normal  data.  In  effect,  Aimitage  and  Siegmund  order  the  points 
on  the  stopping  boundary  in  a  natural  way;  they  then  use  this  ordering  to  construct  a 
family  of  tests  of  hypotheses  of  the  form  9^  do  for  arbitrary  9o',  and  then  they  invert 
this  family  of  tests  to  form  confidence  bounds.  See  also  Siegmund  (1985,  §§3.4,  4.5). 
This  approach  has  attracted  substantial  interest  recently  (Bather,  1988;  Facey  &  White- 
head,  1990;  Kim,  1987).  On  the  other  hand,  it  is  moderately  complicated. 

The  purpose  of  the  present  paper  is  to  present  an  alternative  confidence  procedure, 
which  starts  with  an  approximately  pivotal  quantity  and  then  proceeds  in  natural  ways. 
The  alternative  seems  both  conceptually  and  technically  simpler. 

The  alternative  is  developed  in  the  context  of  a  nonstandard  truncated  sequential 
probabability  ratio  test  about  a  norma!  mean,  although  the  method  is  fairly  general. 
Truncated  sequential  probability  ratio  tests  are  described  in  §2,  and  the  alternative 
approach  in  §  3.  In  §  4  the  results  of  a  simulation  study  of  the  alternative  procedure  are 
reported.  Section  5  contains  some  brief  remarks  on  possible  modifications  of  the  alterna¬ 
tive  procedure  and  its  domain  of  applicability;  §6  contains  a  rough  derivation  of  an 
approximation  to  the  error  probabilities. 

0428”*03389 


Nonpaxametric  Bayes  Estimation 
of  a  distribution  function  with  truncated  data 


Mauro  Gaspaiini^ 
March  20, 1991 
The  University  of  Michigan 


Abstract 

A  truncation  bias  affects  the  observation  of  a  pair  of  variables  (X,Y),  so  that 
data  are  available  only  if  V  <  A*.  In  such  a  situation,  the  nonparametric  majdmum 
likelihood  estimator  (NPMLE)  of  the  distribution  function  of  Y  may  have  unpleas¬ 
ant  features  (Woodroofe  (1985)).  As  a  possible  alternative,  a  nonparametric  Bayes 
estimator  is  obtained  using  a  Dirichlet-Ferguson  prior.  Its  frequentist  asymptotic 
behavior  is  investigated  and  found  to  be  the  same  as  the  asymptotic  behavior  of 
the  NPMLE.  The  results  are  illustrated  by  an  example,  with  astronomical  data, 
where  the  NPMLE  is  clearly  unacceptable. 


^Research  partially  supported  by  U.S.Army  Research  Office  under  DAAL-03-88-0122. 

°Key  words  and  pArases.Nonparametric,  Bayes  estimation,  Dirichlet-Ferguson  prior,  truncated  data, 
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THE  DOUBLY  ADAPTIVE  BIASED  COIN  DESIGN 
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Abstract:  A  biased  coin  design  is  proposed  for  the  allocation  of  subjects  to 
treatments  A  and  B  in  a  clinical  trial  when  the  desired  allocation  proportions 
are  unknown.  The  design  is  doubly  adaptive  in.  the  sense  that  it  takes  account 
of  the  proportion  of  subjects  assigned  to  each  treatment  and  the  current  es¬ 
timate  of  the  desired  allocation  proportion.  A  strong  law  of  large  numbers  is 
established  for  the  proportion  of  subjects  assigned  to  a  treatment  when  sub¬ 
ject  responses  are  independent  random  variables  from  standard  exponential 
families.  The  normal  case  is  presented  as  an  application. 
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A  LOCAL  LIMIT  THEOREM  FOR  PERTURBED  RANDOM  WALKS 
By  Mei  WANG* 
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Abstract:  The  main  result  reported  here  is  a  Stone  type  local  limit  theorem  for  perturbed  random 
walks  Zn=Sn+  Cn  when  some  slow  variation  conditions  are  imposed  on  ^n’s* 

Key  words:  Local  (central)  limit  theorem,  Edgeworth  expansion,  perturbed  random  walks. 

1.  Background 

When  specialized  to  one  dimension  and  non-lattice  distributions,  Stone’s  Theorem  (1965) 
asserts  the  following:  Let  Xi,X2,Xz,‘  ’  •  denote  i.i.d.  non-lattice  random  variables  for  which 
E[Xi]  =  OasidE[Xi^]:^L  Let 

Sn  ~  Xi  -f-  X2  ^ —  *  +  Xin  Tl  =s  1, 2, 3,  •  •  • 

be  the  sums,  called  “random  walks”.  Then  for  each  L  6  [0, 00), 

CbW  =  supsup  v^F{6<  5„  <  6-fc} -c(^( 0> 

c<Xi6JR  \v”/| 

where  ^(®)  is  the  standard  normal  density  and  R  is  the  real  line.  A  consequence  of  this  is 

vSP{S„6J}  — .  Ji 

n-00 

for  any  interval  J  C  jR  of  length  |  J].  This  is  the  result  of  Shepp  (1964). 

There  has  been  recent  interest  in  sequences  of  random  variables,  called  perturbed  random 
walks.  See,  for  example,  Siegmund  (1985),  Woodroofe  (1982),  and  Lalley  (1984).  A 

*  Re»earch  partially  supported  by  the  U.S.  Army  Research  Office  under  DAAL  03-88-K.0122. 
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